I am building a website that compares prices of games from 5 sites. But I have a slight dilemma, should I add all games to a database or should I crawl these sites for every request I get, which would mean that the sites that I use to compare the prices have a lot of traffic from my site. On the other hand, it would be a lot of work to create a database with the price information always up to date. What are your thoughts on this problem and/or do you have other suggestions on how to solve this problem?
You must be logged in to post. Please login or register an account.
It depends I suppose on how many requests you will get, but, personally, the superior choice would be to run a cronjob that parses the sites every hour/day/whatever, and stores the information to a database. Why do you think it would be a lot of work to create a database? If you're pulling the requests dynamically, you've already got the code to do it, so really the only difference is putting it into a database and retrieving it from the database. Not much work at all.
-Harrison 7 years ago
You must be logged in to post. Please login or register an account.